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Detailed measurement of cell phenotype information from digital fluorescence images has the potential to 
greatly advance biomedicine in various disciplines such as patient diagnostics or drug screening. Yet, the 
complexity of cell conformations presents a major barrier preventing effective determination of cell 
boundaries, and introduces measurement error that propagates throughout subsequent assessment of 
cellular parameters and statistical analysis. State-of-the-art image segmentation techniques that require 
user-interaction, prolonged computation time and specialized training cannot adequately provide the 
support for high content platforms, which often sacrifice resolution to foster the speedy collection of 
massive amounts of cellular data. This work introduces a strategy that allows us to rapidly obtain accurate 
cell boundaries from digital fluorescent images in an automated format. Hence, this new method has broad 
applicability to promote biotechnology. 

The accurate determination of cell morphology is critical to many aspects of biomedicine 14 . For instance, 
histopathology can indicate the stage of cancer based on the organization and shape of cells present in a 
patient biopsy; yet, it still relies on experienced physicians to visually recognize the qualitative differences in 
cell phenotype 4,5 . If morphological analysis could be performed quantitatively, the greater potential to reveal 
subtle disparities in cell phenotype could radically improve the way we grade cancer 3 6 . Meanwhile, the drug 
screening industry has actively adopted computer-guided morphological assessment to uncover the potential of 
new drugs 7 " 9 . High content screening (HCS) platforms allow us to gain access to rich phenotypic information that 
can be quantitatively analyzed and statistically distinguished, but the priority of existing platforms is to foster the 
speed of image processing so some measurement resolution is often conceded 7,810 . Therefore, an advance in 
technology that improves our ability to rapidly and accurately quantify cell morphology can greatly impact the 
biomedical community. 

However, the complexity of the fluorescent signal from a typical cell within a digital image presents a major 
barrier for the generation of accurate cell boundaries from segmentation 1215 . The fastest way to generate a cell 
boundary is to compare the pixel values of an image to a single intensity threshold, which may be determined 
quickly using histogram-based approaches 1013 . These kinds of segmentation strategies only roughly approximate 
cell boundaries, and consequently produce a great amount of error in phenotypic parameters that would not 
otherwise be present during subsequent measurement and statistical analysis 5 8 . This diminishes the capacity to 
advance cellular biophysics using detailed morphological information to support the previously mentioned 
applications. In this regard, more sophisticated image processing methods were developed to offer better bound- 
ary resolution, but they often require prolonged computation time, user-interaction or specialize training for 
proper implementation and remain outside of the mainstream 7,8,16 21 . Hence, the development of a quick and 
accurate segmentation strategy to deliver rich cell phenotype information could dramatically advance patient 
diagnostics and drug discovery. 

In a microscope system, the signal from a specimen first exists as emitted photons, which follow a light path to a 
photo-detector to be converted into a digital signal. During this process, several sources of error can obscure the 
true signal and cause a loss of spatial resolution in raw images. Light gathered from a fluorescent specimen is 
subject to the influence of the acquisition systems and its intensity is distributed spatially based on the system 22 . 
The interference from the path as the light passes through the microscope system from the fluorescent specimen 
to the detector makes the microscope essentially act as a physical spatial low pass (SLP) filter and blends fine 
image features, like thin lines or edges, to reduce the local contrast 23 . 
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This work introduces a segmentation scheme that takes advantage 
of the concept of spatial filtering to effectively address segmentation 
pitfalls that arise naturally during image acquisition. The method 
requires minimal user interaction, is computationally inexpensive; 
meanwhile, can produce accurate cell boundaries despite a range of 
image conditions and cell morphologies. Hence, this approach can be 
easily implemented as an add-on for existing open source software 
packages to improve their ability to reliably segment fluorescent cell 
images 10,11 , and can directly promote high content quantitative mea- 
surements of cell features, which are not accurately provided by 
current methods. 

Results 

Intrinsic features of fluorescent cell images impede segmentation. 

Cells expressing fluorescence proteins or dyed by a certain 
fluorescence reagent emit a light signal that varies in intensity 
spatially throughout the cell due to morphological features or 
spatial exclusion by cellular organelles and lipid membranes. 
Intensity profiles of raw images that capture the cell signal also 
contain a variable noise that intrinsically arises during image 
acquisition. In a histogram of the raw pixel intensities, the pixels 
corresponding to the background region of the image typically 
generate a large peak in occurrence at lower intensity values while 
pixels corresponding to the cell signal are distributed broadly among 
the higher intensity values, appearing as a 'tail' (Figure 1 a). 

Segmentation is the process of identifying the pixels that corre- 
spond to the cell signal in an image. For instance, segmentation by a 



single intensity threshold aims to distinguish the cell signal from the 
background by partitioning the image into two classes of pixels 12,13 . 
Setting the threshold near to the peak of the histogram may omit 
much of the background and grant good segmentation for low 
intensity cell features, such as cell protrusions; however, it also 
includes some of the background noise and any background pixels 
that have been influenced by the glare of nearby cellular fluorescence. 
Alternatively, setting a threshold further from the histogram peak 
will omit nearly all background pixels, but may also omit low intens- 
ity regions of the cell signal. 

Closer examination of the intensity profile of a cell image over the 
cell body and one protrusion reveals that the peak intensity value of 
the signal from the protrusion falls below the threshold value that 
could properly segment the cell body region of the image (Figure 1 b). 
This is a natural consequence of the image acquisition process, which 
reveals that the pixels corresponding to the cell boundary in one 
region of an image do not necessarily possess similar intensity values 
to cell boundary pixels in another region of the same image, prevent- 
ing accurate segmentation by a single threshold. 

Furthermore, the intensity profile of a cell image can alter depend- 
ing on the level of fluorescent protein expression, the current shape 
configuration of the cell and/or the parameters of image acquisition. 
For example, the exposure time of the camera can be manipulated to 
allow more or less photons to reach the camera, resulting in images 
with drastically different intensity profiles (Figure 1 c). As exposure 
time is increased, the average intensity value of the image will 
increase and a boundary determined solely by the statistics of the 
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Figure 1 | The light signal from a fluorescent cell sample greatly affects digital image intensity profiles, (a) A histogram of the intensity distribution in a 
typical fluorescent image of a NIH 3T3 fibroblast is shown. The raw image (both inset) was captured using x = 100 ms and 1000 bins were used to 
construct the histogram. A segmentation threshold was calculated using the mixture of Gaussian (MoG) method and plotted as the vertical, red line, 
separating the histogram into a background class (gray) and the remaining pixel intensities. The boundary that the MoG threshold produces is drawn in 
red over the left inset image. A second segmentation threshold was calculated using Otsu's method and plotted as the vertical, blue line. The boundary that 
Otsu's threshold produces is drawn in blue over the right inset image. Arrows indicate important regions of the image that were segmented differently by 
each method, (b) Histograms of cropped images from two regions in the raw cell image are shown. The cropped image of one cell protrusion (left inset) 
produces a histogram (gray) with a background class of pixels and small 'tail' corresponding to the cell protrusion signal. The boundary of a MoG 
threshold is shown. The cropped image of the cell body (right inset) also produces a histogram (black) with a background class and a tail. The boundary of 
an Otsu threshold is shown. The peak intensity (arrow) for the cropped image of the protrusion falls into the background pixel class of the cell body 
histogram, (c) The intensity profile of a fluorescent image changes depending on the exposure time (x) during image acquisition. Images taken at x = 50, 
100 and 300 ms (inset) generated histograms colored in red, black and blue, respectively. As x increases, the average image intensity also increases and the 
background class appears to develop a wider and shorter intensity distribution, (d) The average background intensity of each different exposure was 
subtracted to adjust the intensity of each histogram for comparison. A threshold (vertical red line) was chosen by eye to segment each adjusted image 
(inset). 
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background noise will progressively broaden (Figure 1 d). This effect 
of average image intensity could cause some cells to seem larger after 
segmentation only due to the apparent signal rather than physical 
differences among cells for the aforementioned reasons. Segmenta- 
tion strategies that do not account for this effect might generate 
inaccurate boundaries from cell images, and further introduce error 
into later phenotypic measurements. 

Automated spatial filtering generates rapid and accurate cell 
boundaries. Fluorescent cell images must be subjected to some 
level of processing to avoid the pitfalls inherent in the intensity pro- 
file of raw images and obtain a good segmentation result. Though 
some sophisticated methods exist to segment cell images 16 21 , we have 
developed a new procedure that generates accurate segmentation 
results without the need for high computational power or any 
user- interaction (see Supplementary Figure SI online for compari- 
son). Briefly, a spatial low pass filter is applied to raw images, and the 
resulting low pass result is re-scaled and then subtracted from the raw 
image to produce the final processed image. This section describes 
the process in greater detail. 

In a raw image, edges of objects are visually apparent because they 
possess high contrast from one pixel to the next; this contrast is 
considered to be high spatial frequency content in the image. On 
the other hand, the gradual changes in intensity that prevent good 
segmentation are considered to be low spatial frequency content. In 
plain terms, high spatial frequencies resemble tv static, edges or 
narrow stripes that are close together while low spatial frequencies 
appear as uneven illumination or blurriness. Thus, a potentially 
effective approach to aid segmentation is spatial filtering, which 
allows us to process images by enhancing or reducing only one type 
of spatial frequency content 13,24 . 

For example, sharpening of raw images will enhance the high 
spatial frequencies and improve contrast by making the intensity 
differences between edge pixels more drastic. However, straightfor- 
ward application of a high pass spatial filter also enhances the power 
of the pixel noise, which also varies from pixel-to-pixel. On the other 
hand, a low pass spatial filter can be applied first, and then subtracted 
from the raw image values to achieve sharpening without aggravating 
the pixel noise 12,25 (Figure 2 a). The applied low pass filter must match 
the scale of the object it is meant to reveal through processing and its 
dimensions can be calculated using an initial threshold to approx- 
imate the size of a cell for sharpening applications (see Supple- 
mentary Figure S2 online). However, without further processing, 
such sharpened images often identify contrast within the region of 
cell signal due to variability in cell morphology and could generate 
false edges during segmentation 12,26 (see Supplementary Figure SI 
online). 

This work introduces the key step of re-scaling the low pass 
result before subtraction from a raw image to avoid the issue of 
false edges and benefit segmentation greatly. In a cellular fluor- 
escence image, the intensity profile of the low pass result always 
exceeds the value of the raw image intensity profile at high contrast 
regions in the image (Figure 2 b). After subtraction of the low pass 
result from the raw image, this disparity generates a global min- 
imum pixel value in the high pass image (Figure 2 c). At that pixel 
position, the intensity value of the raw image and the low pass 
result can be extracted to create a re-scaling factor. Then, direct 
multiplication of the re-scaling factor with the entire low pass 
image (subtracting the average background intensity beforehand) 
reduces its intensity values so that its intensity profile can 'fit' 
underneath the original raw intensity profile (Figure 2 d). 
Notably, the raw image and re-scaled low pass result will have 
equivalent intensity at the point of the global minimum. 
Ultimately, subtraction of the re-scaled low pass result from the 
raw image intensity values yields a new image with boundary 
pixels that are nearly within a single plane of intensity (Figure 2 e). 



Inspection of a histogram for the final processed image reveals a 
'dip' between the background peak and cell signal tail, and sub- 
sequent segmentation of the new image reveals excellent agreement 
with the apparent cell boundary (Figure 2 f). Further inspection of 
two cropped regions within the original image demonstrates how the 
respective histograms have automatically been 'adjusted' for differ- 
ences in background level intensity. After processing, the cell signal is 
clearly distinguished by a single threshold in both cropped images 
(Figure 2 g). A closer look at each separate histogram suggests that 
processing has augmented the raw image so that the mean back- 
ground intensity has been adjusted locally due to the subtraction of 
the re-scaled low pass result, greatly facilitating the ease of segmenta- 
tion by a single threshold. 

Automated spatial filtering is robust. Use of this automated 
approach to process images taken at various exposure times re- 
vealed segmentation results that were highly consistent (Figure 3a). 
In addition, application of a de-noising algorithm, such as 
anisotropic diffusion 27 , during the process before segmentation can 
remove any remaining interference from the pixel noise at 
boundaries and generates a highly accurate segmentation result. In 
comparison to boundaries from raw images, boundaries determined 
by this segmentation strategy are highly accurate and consistent 
despite changes in image intensity profiles. 

The boundaries extracted by this method can be directly compared 
to other types of cell images taken at higher resolution to help sup- 
port the segmentation accuracy. Variable pressure scanning electron 
microscopy (vpSEM) was performed on the same exact fixed cells 
that were imaged by fluorescence microscopy. This SEM technique 
does not require surface modification or coatings and was chosen for 
its ability to generate ultra-high resolved images of cells without 
damaging their integrity 28 . The boundary acquired from a fluor- 
escent image was expanded and rotated (an affine transformation) 
to overlay onto the vpSEM image, showing excellent agreement with 
the apparent vpSEM image boundary (Figure 3 b). Similarly, com- 
parison to a differential interference contrast (Nomarski) image of 
the same cell, taken without using the binning feature of the camera, 
showed very close agreement with the visually apparent boundary 
(Figure 3 c). From the cross-comparison, it suggests that the pro- 
posed method could detect subtle differences in cell morphology that 
may have been previously undetectable. 

Discussion 

Even ideally prepared samples, free of debris or out-of-focus objects, 
produce images that still contain segmentation pitfalls as a con- 
sequence of the image acquisition process. Various imaging process 
techniques have been developed to attack this issue from different 
perspectives depending on the training and tools available to indi- 
vidual researchers, each aiming to provide an acceptable measure- 
ment of relevant cellular information. However, image processing 
algorithms that are not capable of effective segmentation without 
prolonged computation or user-interaction only provide a roadblock 
to HCS applications; hence, less precise segmentation strategies are 
chosen to promote speed. Our work presents a straightforward seg- 
mentation strategy to automate the processing of raw images and 
tackle the existing issues that hinder the development of a more 
powerful class of assays based on morphology and localization. 

The developed segmentation strategy possesses several advantages. 
Firstly, it utilizes parameters taken directly from images so that user 
involvement is minimal. Secondly, it is not iterative so that a stopping 
criterion is unnecessary and it can be performed rapidly. Thirdly, this 
method results in consistent boundaries even for heterogeneous 
intensity profiles and for a wide range of signal-to-noise ratios from 
various acquisition conditions. Hence, this method possesses the 
capacity to capture highly dynamic cell behavior, regardless of cell 
types that exhibit a wide range of morphologies and protrusions. 
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Figure 2 | Subtraction of a re-scaled spatial low pass result from a raw image drastically improves subsequent segmentation, (a) Three images are 
shown: a raw fluorescence image (left), the result of applying a spatial low pass filter to the raw image (middle), and the result of subtracting the low pass 
image values from the raw image values (right), which produces a sharpened image, increasing contrast at the edges of the cell (arrows), (b) Selected 
intensity profiles for the raw image (black) and the low pass result (red) are plotted according to the index of the global minimum in the sharpened image. 
Part of the low pass intensity profile exceeds the raw image intensity (arrow), (c) Once the low pass result is subtracted from the raw image values (black), 
the sharpened image intensity profile (red) will contain a global minimum (red arrow). The vertical blue line demonstrates the index of the minimum 
point in the sharpened image, (d) The ratio of values between the raw (black) and low pass (red) image at the pixel index corresponding to the global 
minimum (vertical blue line) is set as a scale factor to reduce the intensity of the low pass result (blue arrows). The scaled low pass result (blue curve) now 
appears to 'fit' below the raw image intensity profile, (e) Subtraction of the scaled low pass result from the raw image values generates a new profile (black) 
drastically improving the ability to perform segmentation by a single MoG threshold (red dotted line). This process treats the index of the global 
minimum (vertical blue line) as part of the background since the value of the final image at that point equals the mean background intensity, (f) After 
processing, the histogram of the final image appears to feature two distinct distributions (arrow). The boundary that a MoG threshold (vertical red line) 
generates is shown (inset), (g) Histograms of two cropped regions in the final image are shown. Both the cropped image corresponding to one cell 
protrusion (left inset) and part of the cell body (right inset) can now be segmented by a single MoG threshold. 



In conclusion, this interdisciplinary work will directly impact the 
transition from qualitative observation to quantitative measurement 
of cell phenotype by not only providing effective segmentation, but 
also fostering automated, high content cellular information. This will 
grant the capability to generate the massive amount of biophysical 
data necessary to identify important morphological subtleties and 
propel biomedicine research into a new era. 

Methods 

Fluorescence microscopy. A Nikon TE-2000 microscope (Nikon, Melville, NY), 
equipped X-Cite 120 PC fluorescent light source (EXFO, Ontario, Canada) and a 



Cascade:lK CCD camera (Roper Scientific, Tucson, AZ), was used to acquire 
microscopic images at 20 X magnification. In addition, an on-stage incubator (In 
Vivo Scientific, St. Louis, MO) with temperature control and a supplementary C0 2 
system was operated along with the microscope to maintain the experimental 
environment at 10% C0 2 and 37°C. 

Cell culture, plasmids and transfection. NIH 3T3 fibroblasts (ATCC, Manassas, 
VA) were maintained in DMEM with 10% fetal bovine serum and 1% L-glutamine 
(all purchased from Mediatech Inc., Manassas, VA) in a humidified incubator at 37°C 
and 10% C0 2 . For transfection, cultured cells were prepared on fibronectin-coated 
glass bottom dishes (MatTek, Ashland, MA) under normal culturing conditions for 
24 hours. Then the cell culture was incubated in 1 ml Opti-MEM I Reduced Serum 
Media (Invitrogen, Carlsbad, CA), which was supplemented with a premixed 
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Figure 3 | Automated segmentation is robust and accurate, (a) Fixed cells imaged using progressively longer exposure times generate progressively 
brighter images. A MoG threshold was used to segment each image and generate cell boundaries (top row) for images taken at x — 50, 100 and 300 ms, 
colored red, white and blue, respectively. Each image was processed using the developed spatial filtering method and segmented by MoG thresholds to 
generate cell boundaries (middle row). Pixel noise remained an issue after processing, especially for dimmer images with less signal-to-noise ratio. 
Application of a de-noise algorithm (anisotropic diffusion) to the processed images before segmentation attenuated the pixel noise and greatly enhanced 
the final cell boundaries, (b) Variable pressure scanning electron microscopy (vpSEM) was performed to image cell samples at a higher resolution than 
fluorescence microscopy is capable without a coating such as gold. Segmentation was performed on raw images of the fluorescence signal (x — 100 ms) 
based on the developed approach. The boundary of that segmentation was scaled and rotated to match the length scale of the vpSEM image and overlaid 
onto it (dotted black line), (c) Dynamic interference contrast (DIC) microscopy was performed to capture an image of the cell sample for further 
comparison. The same boundary generated from analysis of a fluorescence image was scaled to the length scale of the DIC image and overlaid onto it. 



transfection solution, containing 4 ug pERFP-C-RS plasmid (Clontech, Mountain 
View, CA) and 5 ul Lipofectamine 2000 (Invitrogen), for 1 hour to introduce the 
plasmids into the cells. Afterward, the cells were then maintained in normal cell 
culture media for further experiments. 

Variable pressure scanning electron microscopy. Highly resolved cell images were 
acquired by an S-3000N variable pressure scanning electron microscope (Hitachi, San 
Francisco, CA). An acceleration voltage of 5 kilovolts and 10.3-mm working distance 
were used. This type of scanning electron microscopy (SEM) uses a low vacuum 
(60 Pa) to enable the imaging of samples without a special coating such as gold. Cells 
expressing red fluorescence protein (RFP) were passed to glass slides and allowed to 
adhere before being fixed in glutaraldehyde for 10 minutes, washed in phosphate 
buffered saline and stored in de-ionized water. Cell samples were allowed to air dry 
before the vpSEM experiments, and were imaged using fluorescence microscopy 
immediately before and after the vpSEM to allow for direct comparison of the same 
cells between the two techniques. In this case, sample shrinkage was not a concern 
since only the fluorescent signal was of interest; hence, special freeze-drying was not 
used to prepare samples. 

Image analysis and threshold determination. Raw images were processed using a 
custom-designed routine in Matlab (The MathWorks, Natick, MA) as described in 
the results section. 
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